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ABSTRACT 


This  paper  considers  a  class  of  Markov  chains  on  a  bivariate  state 
space  (1N,E),  whose  transition  probabilities  have  a  particular  "block-parti¬ 
tioned"  structure.  Examples  of  such  chains  include  those  studied  by  Neuts 
(1978)  who  took  E  to  be  finite;  they  also  include  chains  studied  in  queueing 
theory,  such  as  (Nn,Sn)  where  Nq  is  the  number  of  customers  in  a  GI/G/1 
queue  immediately  before,  and  SR  the  remaining  service  time  immediately 

a  1 

after,  the  n™ arrival. 

We  show  that  the  stationary  distribution  II  for  these  chains  has  an 
"operator-geometric"  nature,  with  II(k,*)  *  /ll(0,dy)  Sk(y,*)»  where  the 
operator  S  is  the  minimal  solution  of  a  non-linear  operator  equation. 
Necessary  and  sufficient  conditions  for  II  to  exist  are  also  found.  In  the 
case  of  the  GI/G/1  queueing  chain  above  these  are  exactly  the  usual  stability 
conditions. 


Keywords:  Gl/G/1  queue,  phase-type,  invariant  measure,  Foster’s  conditions. 


1.  INTRODUCTION 


In  [8],  [9]  and  [10]  Neuts  has  shown  that  certain  classes  of  Markov 
chains  admit  a  stationary  distribution  of  a  form  which  he  has  termed  "matrix- 
geometric".  Such  chains  have  a  bivariate  state  space  (i,j),  i  *  0,1,..., 
j  »  l,...,n;  the  stationary  distribution  II ( i » j )  has  the  geometric  property 
that 

n(0,j).-  on(j),  j  =  l,...,n, 
n<i,j)  «  (Qns~)j,  j  -  l,  ...,n, 

for  a  particular  n*n  matrix  S. 

The  class  of  chains  with  this  structure  includes  embedded  Markov 
chains  associated  with  GI/PH/1  queues,  where  the  service  time  distribution 
is  "phase-type"  [7].  The  second  variable  j  then  indexes  the  phase  of  the 
service  time. 


In  this  paper  we  investigate  more  general  chains  with  bivariate  space 
(i,x),  where  the  first  coordinate  i  takes  on  integer  values  0,1,..,  but  - 
the  second  coordinate  x  is  in  E,  some  general  measure  space.  In  particular 
examples  such  as  the  GI/G/1  queue  in  §4  below,  E  will  be  taken  as  [0,®). 

We  let  &  denote  o-field  on  E;  when  E  ■  [0,®)  we  will  take  S  as  the  Borel 
o-field,  We  denote  by  IN  the  set  {0,1,...},  and  the  set  i*E  will  be  called 
the  level  i,  and  denoted  by  i. 

The  basic  Markov  chain  {X  }  that  we  shall  study  is  assumed  to  have  a 

n 

transition  probability  law  denoted  by  P(i,x;  j ,A)  for  i,j  c  ft,  x  e  E,  A  e  fi. 
As  usual  it  is  taken  to  be  a  measurable  function  in  x  for  each  A  e  S  and 
a  substochastic  measure  on  &  for  each  i,j  e  ft  and  x  e  E;  we  also  assume 


\  P(i,x;  j,E)  *  1.  The  n-step  iterates  of  P  are  defined  by 

1 


Pn(i,x;  j ,A)  -  l  f  Pn'1(i,x;  k,dy)  P(k,y;  j,; 
v  > 


and  have  the  interpretation 


-  2  - 


Pn(i,x;  j,A)  “1pfXn  e  j*A  I  XQ  -  (i,x)}  . 

We  now  impose  Che  following  "spatial  homogeneity"  pattern  on  the 
transition  law  P.  We  assume  that 

P(i,x;  j,E)  -  0  ,  j  >  i  +  1  , 


and  for  each  k  *  0,1,...  there  exists  a  substochastic  transition  kernel  A^ 
(that  is,  a  collection  A^(x,A),  x  e  E,  A  e  £,  measurable  in  x  for  each  A 
and  a  measure  on  £  for  each  x  e  E,  with  A^(x,E)  <_  1,  x  e  E)  such  that  for 
j  J4  0, 

(1.1)  P(i,x;  j,A)  -  Ai_j+1(x,A)  ; 
and  we  write 

P(i,x;  0,A)  -  B^x.A)  . 

Since  P  is  stochastic,  from  (1.1)  we  have 

CO 

P(i,x;  0,E)  -  1  -  l  A.  . (x,E) 
j«l  1 

i 

”1-1  A  (x.E) 
j-0  3 

(1.2)  *  Bi(x,E)  . 


Dlagramatlcally,  (1.1)  and  (1.2)  show  that  we  can  write  P  as 


(1.3) 


B0  A0  ®  *  *  * 

B1  A1  A0  ® 

B2  A2  A1  A0  ® 


%••••• 

V  •  •  ■  •  •  * 

\  •  •  •  •  • 

by  labelling  the  states  in  increasing  order  on  the  first  variable.  Typically 

OO 

(see  [8]  and  §4)  in  queueing  applications  we  have  £  A^(x,E)  *  1, 


k-0 


B .  (x.E)  -  l  A.  (x.E). 
J  j+1  * 


so  that 


2.  THE  OPERATOR-GEOMETRIC  FORM  OF  H 


A  stationary  probability  measure  II  for  the  chain  {X  >  is  a  collection 

n 

of  measures  n(j,*)  on  E,  j  e  N,  wirh  £  II(j,E)  -  1,  and  satisfying 

3 

n(J.A)  -  l  (  n (k,dy)  P(k,y;  j,A)  ,  A  €  £,  j  e  &  . 

k  ' 

K  E 

From  (1.1)  and  (1.2),  for  j  >  0 

(2.1) 

whilst  for  j  ■  0, 


n(j,A)  -  1  J  n (k,dy)  A^^y.A) 

k  E 


(2.2) 


n(0,A) 


l 

k 


J  II(k,dy)  Bk(y,A)  . 
E 


In  order  to  use  the  theory  of  general  Markov  chains  to  find  conditions 
under  which  a  measure  H  satisfying  (2.1)-(2.2)  exists,  we  shall  assume  the 
following. 

Irreducibility  condition:  There  is  a  measure  $(•)  ontlxE  such  that,  when¬ 
ever  .  $(j, A)  >  0  for  some  j  e  N,  A  e  &,  for  each  i  e  to  and  x  e  E  we  have 


(2.3) 

and 


0  <  l  Pn(i,x;  j , A)  ; 
n“l 


(2.4)  <K0,E)  >  0  . 

The  condition  (2.3)  is  just  the  standard  4>-irreducibility  condition  (see 
[14],  §2,  ot  [11],  [12]).  We  impose  (2.4)  specifically  because  of  the 
behaviour  we  wish  to  investigate  :  the  chain  satisfying  (2,4)  and  (2.3) 

is  such  that  from  any  pair  (i,x),  the  zero-level  0  can  be  reached  eventually 
with  positive  probability. 

The  key  to  the  operator  geometric  structure  of  any  solution  II  to 
(2.1)  and  (2.2)  is  a  last  exit-time  representation  of  n,  familiar  in  the 
countable  case  (cf.  [1]).  We  write,  for  i,j  e  to,  x  e  E,  A  «  S, 


-  4  - 


-P  (i,x;  j , A)  - 


•)Pn(i,x;  j,A)  -  l 
■  k>0 


,n-l 


P(i,x;  j,A) 

(i,x;  k,dy)  P(k,y;  j,A)  . 


These  are  che  taboo  probabilities  of  (Xn)  where  the  level  0  is  taboo;  and 
we  write 

CD 

L(i,x;  j,A)  -  l  pn(i,x;  j,A)  . 
n-1  U 


If  Trt  *  inf (n  >  0  :  X  e  0) ,  then 
u  n 

L(0,x;  0,E)  "3P(t0  <  -  )  XQ  -  (0,x)) 
1  1  » 


and  L(0,x;  j,A)  for  j  >  0  is  just  the  expected  number  (possibly  infinite) 
of  visits  to  state  j  *A  before  returning  to  level  0. 

We  now  have 

Theorem  1  ;  (i)  If  the  chain  (X^}  has  a  stationary  probability  measure  H, 

then  n(0,E)  >  0.  For  n-almost  every  x  c  E,  we  have  L(0,x;  0,E)  *  1. 

The  chain  {„X  }  defined  on  E  with  transition  law  L(x,A)  *  1(0, x;  0fA)  has 
o  n 

a  stationary  probability  measure,  qJI(’),  satisfying 

(2.5)  0n(A)  =  J  QH(dy)  L(y,A) ,  A  e  £  . 

E 


The  measure  H  is  then  given,  for  j  >  0,  by 


(2.6) 

H(j,A)  ■  c  f  gll(dy)  L(0,y;  j,A), 

and 

/ 

E 

(2.7) 

E(0,A)  »  C  qII(A)  ,  A  e  &  , 

where 

(2.8) 

C  -  CH(0,E) ]-1  >  0  . 

(ii)  If  qII(A),  A  e  £  is  a  probability  measure  on  £  satisfying 
,  then  II  defined  by  (2.6)  and  (2.7)  is  a  o-finite  measure  satisfying 


(2.5) 


(2.1)  and  (2.2). 

Proof:  Most  of  these  results  are  veil-known  in  a  general  context.  If  II 

exists  as  in  (i) ,  then  H  »  $  and  so  IT(0,E)  >  0  from  (2.4).  The  existence 

of  51  implies  the  chain  {x  }  is  1-positive  recurrent  (see  [14])  and  so,  since 

n 

51(0, E)  >  0,  we  have  L(i,x;  0,£)  ■  1  for  II-almost  all  (l,x)  e  IN  *E.  It  is 

then  easy  to  see  ([11],  pp.  32-33)  that  qII(*)  =  11(0,  *)/n(0,E)  is  invariant 

for  the  chain  with  transition  law  L(x,A);  one  has  to  check  that,  writing 

N  *  {y  e  E  :  L(y,E)  <  1),  the  set  N  =  (y  e  E  :  £  Ln(y,N)  >  0}  also  has 

n 

II (N)  *  0,  which  is  a  standard  result. 

Since  (X  }  is  a  ♦-irreducible  chain,  II  is  unique;  it  is  not  difficult 
n 

([11],  p.  32)  to  check  that  the  measure  given  by  the  right-hand  sides  of 
(2.6),  (2.7)  is  invariant  for  (X^),  and  so  must  be  H.  This  also  shows 
that  (ii)  is  true.  □ 

This  theorem  basically  shows  that  the  only  invariant  measure  is 
given  by  (2.6)  and  (2.7),  and  motivates  the  investigation  below  into  the 
structure  of  the  quantities  L(l,x;  J  ,A) .  We  shall  define,  analogously 
with  the  quantities  L(i,x;  Q,A),  the  more  general  taboo  probabilities 

L(i,x;  j ,  A)  -  l  Pn(i,x;  j,A)  , 
n 

where 

APn(i,x;  j,A)  «  P(Xn  ejxA,  Xr^*,r*  l,...,n-l  |  XQ  «  (i,x))  . 

The  first  crucial  aspect  of  the  structure  (1.3)  of  P  that  we  use  is  that 
the  chain  moves  in  a  "right-continuous"  way  between  levels;  from  i  one  can 
Increase  levels  to  i  +  1  but  not  in  one  step  to  i  +  k,  k  >  1.  Hence,  if 
1  <  i,  t  <  j,  £pn(i,x;  j,a)  -  P{Xn  e  j  x  A,  Xr  £  (0,1, ...,A}  r  *  1,..., 
n  -  1  |  Xq  ■  (i,x)}.  We  can  thus  define,  as  in  [9], 

(2.9)  S(k)(x,A)  -  tL(i,x;  i+k, A),  i  eta,  x  £  E,  A  £  S, 

(k) 

independent  of  i.  (This  can  be  checked  by  writing  out  S  in  terms  of 


-  6  - 

the  Aj ) .  We  write 

S(x,A)  -  S(1)(x,A)  . 

The  following  result  then  parallels  Theorem  1  of  [9]. 

Theorem  2:  -Suppose  the  Markov  chain  {X^}  has  an  invariant  probability 
measure  II. 

(i)  For  k  >  0, 

n(k,A)  -  c  |  Qn(dy)  S(k)(y,A)  , 

E 

k 

where  c  ■  [11(0, E)  ] 

(ii)  For  k  _>  0, 

S(k)(x,A)  =  Sk(x,A),  x  €  E,  A  e  &, 

where 

S^(x,A)  *  S(x,A),  x  e  E,  A  e  £, 

and 

Sk(x,A)  -  Sk  ^(x,dy)  S(y,A),  x  €  E,  A  e  £, 

E 

are  the  usual  Iterates  of  the  kernel  S(x,A). 

(iii)  The  kernel  S  satisfies  the  non-linear  operator  equation 

(2.10)  S  -  A[S] 

where  we  .define,  following  [8],  the  formal  series 

00  . 

D[Q] (x, B)  -  l  Qk(x,dy)  D.  (y,B),  x  e  E,  B  «  £. 

k“°  i 

for  any  set  Dj  of  kernels  and  the  iterates  Q  of  any  kernel  Q. 

(iv)  The  kernel  S  is  1-transient  (see  [14])  and  in  fact  for  0IT-almost  all 
x  e  E, 

(2.11)  l  Sk(x, E)  <  ®  . 

k 

(v)  If  S  is  another  kernel  satisfying  S  *  A[S],  then 


7 


(2.12) 


S(x,A)  >  S(x,A),  x  £  E,  A  e  S. 


Proof :  (i)  This  is  just  a  restatement  of  the  general  result  in  (2.6)  and 

(kl 

(2.7),  using  the  fact  that  by  definition  (2.9),  L(0,y;  k,A)  »  S''  '(x,A). 

(ii)  We  have,  decomposing  over  the  time  of  last  entrance  to  level  k. 


nPn(0,x;  k+l,A)  -  l  ' 
u  r=0 


0 


P  (0,x;  k,dy)  kPu  (k,y;  k+l,A) 


(2.13) 


l  (  flPr(0,x;  k,dy)  Pn-r(0,y;  1,A)  , 
r-0  i  u  u 


and  summing  (2.13)  over  n  gives  the  result 


S(k+1)(x,A)  -  J  S°°(x,dy)  S(y,A) 

E 

as  required. 

(iii)  In  this  case  we  decompose  ^Pn  over  the  position  at  time  n-1. 
By  construction,  QP(0,x;  1,B)  -  AQ(x,B)  and  for  n>l, 


(2.14) 


Pn(0,x;  1, B) 


?  1 

k-1  l 


,P  ( 0 ,  x,  k.dy)  Ak(y,B) 


summing  over  n  leads,  using  (ii),  to  the  required  result. 

40 

(iv)  Since  Sk  -  S(k),  we  have  that  £  Sk(x,E)  **  E(x.  )  X.  -  (0,x)). 

k=l  U  U 

It  is  well-known  that,  since  $(0,E)  >  0,  this  quantity  is  finite  for  II- 
almost  all  x  e  E;  to  see  this  one  uses  the  fact  that,  from  (i)  above, 

1  =  n(tf*E)  ■  c  |  Qll(dy)  l  Sk(x,E)  . 

E  k 

(v)  As  in  [9],  we  first  define  another  sequence  of  kernels  by 
setting  Xq(x,B)  =  0,  and  Xj^  *  ACX^]  for  N  >_  0.  For  every  x  e  E,  B  £  & 
we  have  Xq(x,B)  <_  S(x,B),  and  by  induction 


(2.15) 


Xf}+1(x,B)  -  AtXfJ](x,B)  ^  A[SKx,B)  -  S(x,B)  . 


A  similar  calculation  also  shows  XN(x,B)  to  be  monotonically  increasing  in 

N  for  every  x  e  E  and  B  e  &.  Hence  the  quantity  X^x.B)  =  lim  t  X^x.B)  exists 

N-*» 

for  every  x  e  E  and  B  e  £.  For  each  fixed  x,  this  setwise  convergence  guaran¬ 
tees  that  X*(x,*)  is  a  measure  on  S;  for  each  B  e  ,  X*(*,B)  is  clearly  a 
measurable  function  on  E  which  is  finite  except  perhaps  on  the  set 
Ng  -  {y  e  E  :  S(y,E)  *  “}.  The  kernel  X*  is  a  solution  to  (2.10),  since 
the  monotone  convergence  of  X^(x,B)  to  X*(x,B)  implies  firstly  that 

X^(x,B)  +  Xk(x,B),  and  then  that  X*(x,B)  -  lim  X^x.B)  -  A[X*](x,B)  . 

N 

Moreover  (2.15)  shows  that  X*(x,B)  £  S(x,B)  for  any  solution  S  satisfying 
(2.10),  so  that  XA  is  the  minimal  solution  of  (2.10). 

We  now  show  that  in  fact  X^  *  S,  as  in  [9].  Define 

N 

SM(x;  k.B)  -  l  nPn(0,x;  k.B)  , 

N  n-1  ° 

so  that  clearly  SN(x;  1,B)  +  S(x,B).  Now  from  (2.14),  we  have 

(2.16)  SN(x;  1,B)  -  Aq(x,B)  +  £  f.S^U;  k,dy)  A^y.B)  . 

k“X  E 

Moreover,  the  last  exit  equations  (2.13)  give 


SN-1(x;  k+l,B)  <_  j  SN_1(x;  k,dy)  SN_1(y;  1,B),  B  e  &  , 
E 

so  that  SN_1(x;  k+l,B)  <  Sk*J(x;  1,B),  B  e  &. 

Substituting  in  (2.16)  gives 


(2.17)  SN(x;  l.B)  <  l  | 


Finally,  note  that  S^x;  l.B)  -  AQ 
have  by  induction 


Sk_1(x;  l,dy)  A^y.B)  . 

(x,B)  *  X^(x,B),  and  so  from  (2.17),  we 


SN(x;  l.B)  £  Xn(x,B)  . 


Taking  limits  as  N 


gives  S(x,B)  £  X*(x,B)  as  required. 


a 


This  result  shows  that  Theorem  1  of  [9]  and  Theorem  3  of  [8]  are 

essentially  a  product  of  the  standard  construction  of  II  as  in  Theorem  1 

Ic 

above,  with  the  geometric  part  S  coming  from  the  right-continuity  and 
translation  invariance  of  P.  The  multiplier  qII  is  shown  in  Theorem  3  of 
[8]  to  be  an  eigenvector  of  the  matrix  B[S];  our  next  result  gives  the 
probabilistic  explanation  for  this. 

Proposition  1  :  The  Markov  chain  {qXq}  has  the  transition  law 

L(x,A)  -  B{SKx,A) 

and  hence  when  {Xq}  has  a  stationary  measure  II,  for  0II-almost  all  x  e  E, 


(2.18) 


B[S] (x,E)  -  1  . 


Proof  :  As  in  (2.14),  QP(0,x;  0,A)  **  BQ(x,A)  and  for  n  >  1 

0Pn(0,x;  0,A)  -  f  |0Pn"1(0,x;  k,dy)  Bk(y,A)  , 
k-1  ' 

so  that  the  form  of  L(x,A)  follows  on  summing  over  n.  The  result  (2.18) 
is  then  immediate  from  Theorem  l(i).  □ 

The  probabilistic  interpretation  here  explains  (2.18)  which  is 
derived  directly  in  [8].  The  direct  derivation  as  in  [8]  does  give  us 
the  following  uniqueness  result  for  the  kernel  S. 

A  A  A 

Theorem  3  :  Suppose  S  satisfies  S  “  A[S].  For  any  a  e  E,  either 

a  £  N  ■  {y  :  £  Sk(y,E)  *  «},  or  S(a,A)  =  S(o,A),  A  c  5. 
k 

Proof  :  From  (2.12),  S(x,A)  j>  S(x,A)  for  all  x  e  E,  A  e  &  (note  that  this 
proof  does  not  require  H  to  exist) ;  and  so  for  all  A, 


(2.19) 


B[S](x,A)  >  B[S)  (x,A)  ; 


moreover,  for  any  x.  A,  the  inequality  in  (2.19)  is  strict  if  there  exists 
k 

B  with  S  (x,B)  >  S  (x,B)  and  Bk(y,A)  >  0,  y  e  B.  Now  note  from  (1.2)  that 


B^y.E)  B^(y,E)  for  all  y  €  E;  and  so,  since  $(0,E)  >  0,  it  follows  that 
B^(y,E)  >  0  for  all  y  e  E,  from  (2.3).  Hence  from  (2.19)  if  for  some  x  c  E 
and  B  e  £,  S  (x,B)  >  S(x,B),  then 

(2.20)  ^  B[S] (x,E)  >  B[S] (x,  E)  . 

Now  suppose  a  f  N.  Then 

00  , 

B(S](x,E)  -  l  Sk(x,dy)  B  (y.E) 

*  l  [  Sk(x,dy)  [1  -  l  A„ (y,E) ] 

k-0  J  2.-0 

CO  GO  OO  f 

*  l  Sk(x,E)  -  I  I  Sk(x,dy)  A. (y.E) 

k»0  £-0  k-£  ^ 

00  /  OO  00  , 

(2.21)  -  l  Sk(x,E)  -  l  Sk(x,dw)  l  S£(w,dy)  A. (y.E) 

k»0  l  k=0  £-0 

E  E 

00  ,  OO  A 

-  T  Sk(x,E)  -  y  Sk(x,dw)  S(w,E) 

k-0  I  k-0 

E 

-  i 

from  (2.10)  and  the  fact  that  S^(x,E)  =  1.  Since  a  |  N,  the  minimality  of 
S  Implies  a  |  H  so  (2.21)  also  shows  B(S](a,E)  -  1.  Hence  (2.20)  cannot 

.  A  A 

hold  with  a  4  N,  and  so  S(a,B)  =  S(a,B)  as  required.  □ 

This  result  is  in  some  sense  an  analogue  of  Theorem  2  of  [8],  although 
the  proof  is  quite  different,  and  the  uniqueness  result  is  not  as  strong. 

Ue  need  more  structure  than  is  generally  available  (unless  E  is  finite)  to 
gain  the  complete  analogue  of  Theorem  2;  ve  now  investigate  this.  Following 
(15],  we  call  the  kernel  {S(x,A)>  R -recurrent  if  (i)  S  is  y-irreducible  for 
some  y,  (ii)  for  any  r  <  R  there  is  some  A  e  £  with  y(A)  >  0  and  some  x  6  E 

with  y  Sk(x,A)rk  <  (iii)  for  all  A  e  £  with  y(A)  >  0,  and  all  x  €  E,  the 

k 

t*  lc  !c 

series  £  S  (x,A)R  diverges.  Recall  from  (15]  that  there  is  then  a  unique 
k 
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R-subinvariant  measure  Q{*)  satisfying 

(2.22)  x  Q(A)  >_R  J  Q(dy)  S(y,A),  A  e  £, 

which  is  R-invcarumt,  i.e.  satisfies  (2.22)  with  equality.  The  quantity 
R-^  is  the  natural  analogue  of  the  Perron-Frobenius  eigenvalue  for  finite 
matrices,  with  Q  the  corresponding  eigenvector. 

Hence  the  restriction  in  [8]  to  solutions  of  (2.10)  with  spectral 
radius  sp(S)  <  1  corresponds  to  restricting  ourselves  to  solutions  of  (2.10) 
with  radius  of  convergence  R  ^  1.  If  S  is  1-recurrent  then  this  is  the 
natural  analogue  of  sp(S)'  *  1,  whilst  if  S  is  R-recurrent  for  R  >  1, 
then  this  is  analogous  to  sp(R)  <  1.  Our  next  result  thus  covers  the 
dichotomy  of  Theorem  2  of  [8]  exactly  if  we  recall  that  when  E  is  finite  and 
R  >  1,  then  £  S  (x,E)  <  <*>  for  all  x;  this  does  not  happen  when  E  is  infinite. 

Theorem  4  :  Suppose  that  for  some  measure  ifi  the  kernel  A(x,B)  =  ][  A^(x,B)  is 

k 

^-irreducible,  and  that  the  minimal  solution  of  S  »  A[S]  is  u-irreducible 
for  some  p.  Then  the  convergence  norm  R  of  the  kernel  S  satisfies  R  _>  1. 
Suppose  further  that  S  is  R-recurrent.  Either 

(i)  R  *  1:  and  then  S  is  the  unique  (up  to  definition  on  a  \|>-null  set) 
solution  of  S  *  A[S]  with  convergence  norm  at  least  unity;  or 

(ii)  R  >  1:  and  then  if  S  *  A[S],  either  i/KN)  “  0  and  so  from  Theorem  3 

S  =  S  ty-a.e.;  or  N  ■  E,  and  so  S(x,E)  <  S(x,E)  for  every  x  |  N.  In 

particular,  if  {XQ}  has  a  stationary  measure  II  then  either  S  2  S  i|>-a.e.  or 

S  <  S  Qll-a.e. 

Proof  :  Since  (A(x,»)}  is  ^irreducible,  It  has  at  least  one  1-subinvariant 
measure  v  (see{15]).  Let  be  the  sequence  of  kernels  approximating  S  as 
in  the  proof  of  Theorem  2(v).  Clearly  v(B)  J  v(dy)  Xg(y,B),  and  by 
induction 
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(2.23) 


|  v(dy)  X^Cy.B)  “  j  v(dy)  [  |  X^(y,dw)  Ak(w,B) 
E  E  k  E 


<  V(B)  ; 


hence  v(B)  _>  j  v(dy)  S(y,B),  and  hence  [15],  since  S  has  a  1-subinvariant 

measure,  S  must  have  convergence  norm  R  >_  1.  Assume  now  that  S  is  R- 
recurrent. 

(i)  Lee  S  be  any  other  kernel  satisfying  S  *  A[S],  If  S  is  1— recurrent, 

A  AW  A  A 

and  S  has  convergence  norm  R  ^  1,  then  S  S  implies  that  S  is  1-recurrent 

A  A 

also.  Let  Q  be  the  unique  1-invariant  measure  of  S;  from  (2.22),  and  (2.10), 


(2.24) 


Q(B) 


|  Q(dy)  S(y,B) 
E 


Sk(y,dw)  Ak(w,B) 


-  |  Q(dw)  l  Ak(w,B) 


Hence  Q  is  1-invariant  for  the  kernel  (A(x, •))  and  from  (2.23),  we  have 

Aw 

that  Q  is  1-subinvariant  for  S.  Since  S  is  1-recurrent,  it  has  a  unique 

/s 

1-subinvariant  measure  Q  which  is  thus  Q.  Thus  we  have  for  all  B  e  £ 

(2.25)  j  Q(dw)  S(w,B)  -  Q(B)  -  J  Q(dw)  S(w,B)  , 

E  E 

and  from  (2.25)  and  the  minimality  of  S  it  follows  that  for  each  B  e  £, 
S(w,B)  *  S(w,B) ,  except  perhaps  for  w  e  Dg  where  Q(Dg)  m  0.  By  taking  a 
sequence  Bj  t  E  with  Q(Bj)  <  •  for  each  j  it  follows  that  there  exists 
D*U0  with  Q(D)  -  0  such  that  S(w,B)  =  S(w,B),  w  {  D.  Since  Q  »  ^ 

j  ®j 

we  have  proved  (i) . 
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(ii)  As  in  [8],  we  write  A*(x,B)  -  £  A.  (x,B)  zk,  0  <  z  <  1. 

2  k-0  K  ~  — 


Suppose  that  S  is  R-recurrent  with  R  >  1,  and  let  Q  be  the  unique  R- 


invariant  measure  £or  S.  As  in  (2.24)  we  have 


(2.26) 


Q(B)  ■  R  J  Q(dv)  l  R_k  Ak(w,B) 


so  that  Q(*)  is  also  R-invariant  for  A*  (w,A).  Write  C(w,A)  *  A*  (w,A) ; 

R_1  R"1 

and  note  that,  since  {A(x,*)}  is  ^-irreducible  so  is  (C(x, *)}. 

A  /y 

Now  suppose  S  satisfies  (2.10),  and  assume  that  tKN)  >  0*  Note  first 

of  all  that  N  O  {y:  £  S^(y,N)  >0},  and  hence  F  **  Nc  is  "closed"  in  the  sense 
k 


that  if  y  e  F,  then  S(y,N)  *  0.  But  from  Theorem  3,  on  F  we  have 
S(y,*)  =  S(y,*);  so  S(y,N)  “0,  y  £  F.  Since  S  is  y-irreduclble,  this 
implies  that  either  F  is  empty  (i.e.  N  ■  E  as  required)  or  else  y(F)  >  0; 
and  in  the  latter  case  from  (2.22)  we  have  Q(F)  >  0. 

Let  Q^(A)  *  Q (A  n  F);  standard  results  give  that  is  also  R- 

invariant  for  S,  whan  S  is  R-recurrent  (but  not  necessarily  otherwise), 
and  so  from  (2.26), 


(2.27) 


QX(A)  -  R 


J 


Qx(dw)  C(w,A)  . 


But  iterating  (2.27)  and  summing  shows 

Q1(A)  [1-R-1]-1  -  f  Qx(dw)  l  Ck(w,A)  ; 

'  k 

A  A 

taking  A  *  N  then  leads  to  a  contradiction  since  ip(N)  >  0  implies 
\  Ck(w,N)  >  0  for  all  w  but  Q,  (N)  ■  0  by  construction.  0 


Remark  :  If  we  assume  that  E  is  finite  then  the  assumption  that  A  is 
irreducible  is  enough  to  ensure  that  S  is  irreducible,  using  Perron- 
Frobenius  theory:  see  [8],  pp.  187-188. 


§3  Neuts  "mean-drift"  condition 


In  Theorems  2  and  3  of  [8],  Neuts  shows  that  the  existence  for  {X  } 

n 

of  an  Invariant  probability  measure  of  matrix  geometric  form  is  equivalent, 

when  the  matrix  A  ■  J  A-  is  stochastic  and  irreducible,  to 

k  * 


(3.1) 


v(dw)  8(w)  >  1  , 


where  B(w)  ■  £  n  A  (w,E)  and  v  is  the  unique  invariant  measure  for 

n-0  n 

{A(x, •)).  In  his  context,  where  E  is  finite,  v  is  guaranteed  to  exist  from 
Perron-Frobenius  theory,  and  the  proof  that  (3.1)  is  equivalent  to  the 
existence  of  a  stationary  distribution  relies  heavily  on  that  theory. 

In  Che  general  context  we  need  other  tools.  We  have  already  shown 
that  any  stationary  measure  must  be  "operator  geometric";  in  this  section 
we  investigate  the  probabilistic  significance  of  (3.1)  and  show  that  it  is 
very  closely  related  to  the  positive  recurrence  of  {Xq},  provided  B^x,*) 
has  suitable  structure.  Specifically,  we  let  {X^}  be  a  chain  as  in  §1  with 
the  zero-level  probability  structure 


(3.2) 


Bi(x,A)  *  £  A. (x,A)  , 

j-k+1  3 


and  let  Pn(i,x;  j,B)  denote  the  n-step  transition  probabilities  of  {X  } . 

n 

We  shall  need  the  following  relationships  between  {X^}  and  (A(x, •)}. 

Proposition  2  :  (i)  For  every  n,  i  and  x 

(3.3)  Pn(i,x;  fc*B)  -  An(x,B)  . 

(ii)  If  {XR}  is  ^-irreducible,  then  {a(x,*)}  is  ^-irreducible. 


when  'J>(B)  -  *(N  *B),  B  e  &. 


(iii)  If  (X  }  is  4>-irreducible,  and  has  a  stationary 
n 

probability  measure-lT,  then  (A(x,*)}  has  a  unique  stationary  probability 
measure  v,  with  v(*)  ■  II(H  *•'. 
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(iv)  If  (Xn)  is  $ -irreducible,  and  (A(x,*)}  has  a  stationary 


probability  measure,  then  either 

(a)  {X  }  also  has  a  stationary  probability  measure;  or 

n 

(b)  for  ^-almost  all  (i,x),  and  any  level  j. 


(3.4) 


lim  —  l  Pm(i,x;  j,E)  -  0  . 


n-+®  m**l 


Proof  :  (i)  From  (3.2)  we  have  P(i,x;‘  lN,B)  ■  A(x,B),  for  all  i;  and  thence 
by  induction 


Pn(i,x;  N,B) 


-1(i,x;  k,dw)  P(k,w;  (N,B) 
(i,x;  lJ,dw)  A(w, B) 


E 

-  An(x,B)  . 

(ii)  If  5(Hxb)  >  0  then  for  each  (i,x),  there  exists  n  such  that 
0  <  Pn(i,x;  lN,B)  -  An(x,B)  so  that  {A(x, •))  is  ^-irreducible. 

(iii)  If  5  exists,  then  from  (3.3) 


ft(N  x  B)  -  l 

k 


il(k,dw)  P (k, w;  6S,B) 


-  j  fi(H  X  dw)  A(w, B)  , 

and  so  v(«)  ■  fi(N  x  •)  is  stationary  for  {A(x,*)}:  the  uniqueness  of  v  follows 
since  (A(x,*)}  is  ^-irreducible. 

(iv)  Suppose  that  v  exists  but  H  does  not.  From  Proposition  4.2  of 
[14]  there  is  then  a  sequence  of  sets  B(k)  +  fcJ  *  E  such  that  for  all  (i,x), 

Pn(i,x;  B(k) )  -*■  0,  n  -►  ®  . 

Fix  the  level  j  and  let  (j  »c(k))  -  B(k)  •'">  j.  Since  V  is  a  probability 
measure  we  can  certainly  choose  k  sufficiently  large  that  v(C(k))  _>  1  -  E, 
for  given  e  >  0.  Now  Proposition  4.2  of  [14]  also  shows  that  for  ^-almost 
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all  x, 

i  n 

(3.5)  .  lim  -  l  Am(x,C(k))  ■*  v  (C(k))  >  1  -  e  , 

n-*»  n  1 

and  so  from  (3.J)  and  (3.3)  we  have 

lim  i  l  Pm(i,x;  j,E) 
n-+®  n*l 

-  lim  J  l  Pm(i,x;  j,C(k))  +  11m  ^  £  Pm(i,x;  j,E\C(k)) 

n-*®  m«l  rr+®  m*l 

i  Q  ~  i  n 

<_  lim  —  l  Pm(i,x;  B(k))  +  lim  ±  £  Am(x,E\C(k)) 

n-»®  n  m»l  nr*®  n  m*l 

1  e  • 

hence  (3.4)  holds.  □ 

Proposition  2  (iv)  shows  that  when  V  exists  the  level  sets  j  are 
"status  sets"  for  {X  };  see  [14],  §5.  We  need  this  result  in  order  to 

Q 

prove  the  sufficiency  of  (3.1)  for  the  existence  of  a  stationary  measure 

for  {X  }. 
n 

Theorem  5  :  Suppose  {X  }  is  4^-irreducible.  If  A(x,»)  admits  an  invariant 
probability  measure  v  such  that  (3.1)  holds,  then  (X^}  admits  a  stationary 
probability  measure  n. 


Proof  :  Suppose  (3.1)  holds.  It  follows  that 


(3.6) 


0  <  [  V(dw)[l  -  l  £A,(w,E)] 

~  )  £,*0 

E 

r  k  ® 

-  lim  4-  V(dw)[  l  [1-Z]  A.(y,E)  -  k  J  Ap(y,E)]  . 
k-*®  l  Z-0  K  Z-k+1 


Now  write  the  mean  change  of  level  of  {X  }  as 

n 


U(k,w)  -  l  P(k,w;  j,E)  j  -  k  ; 

j-o 

clearly  |u(k,w)  |  <  d  +1  if  k  <_  d,  and 
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(3.7) 


y(k,w)  <  y(d,w)  ,  w  e  E,  k  >  d  . 


From  (3.6),  we  can  choose  d  sufficiently  large  that 

(3.8)  |  v(dw)  y(d,w)  <  0  . 

-  E 

If  we  write  D  -  (y  :  y(d,y)  >  -a),  then  |y(d,w)|  <  a+1  on  D  and  by  choosing 
<*  —  a 

a  sufficiently  large,  from  (3.8)  we  can  ensure 


(3.9) 


J- 


(dw)  y(d,w)  <  0  . 


Now  consider,  for  fixed  i,x,  the  quantity 

♦  (i,x)  -  i  l  [  l  Pm(i,x;  j ,dw)  y(j,w) 
tn-1  £  j 

n  d-1 

(3.10)  -i  [  l  Pm(i,x;  j  ,dw)  y(j,w) 

m-l  l  j-0 

+  r  I  [  l  Pm(i»x;  j.dw)  y(j,w)  . 

n  m-l  J.  j-d 

Suppose  {Xq}  does  not  admit  a  stationary  measure.  From  (3.4)  and  the 
boundedness  of  y(j,w)  for  j  <_  d  we  have  that  the  first  term  on  the  right  of 
(3.10)  tends  to  zero  as  n  +  «,  and  so,  using  (3.7), 

.  n  r  ® 

lim  inf  <b  (i,x)  <_  lim  inf  —  £  £  Pm(i,x;  j.dw)  y(d,w) 

n-*®  n->®  m-l  ^  j-d 

inf  ^  £  £  Pm(i,x;  j.dw)  y(d,w)  . 

*»  n  m-l  I  j-d 


(3.11) 


<_  lim 
n-*® 


But  from  (3.3),  we  have  for  any  B  e  &  and  any  i 

(3.12)  J  l  Am(x,B)  -  if  J  Pm(i,x;  j,B)  +i  f  f  ?"(i,x;  j,B)  ; 
»-l  m-l  j-d  m-l  j-0 


we  have  already  shown  that  the  second  term  in  (3.12)  tends  to  zero,  so  using 
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the  boundedness  of  y(d,w)  on  Dfl  we  have,  putting  (3.12)  into  (3.11). 

l  n  f 

(3.13)  lim  inf  <j>n(i,x)  £  lim  inf  —  ][  Am(x,dy)  ]j(d,w)  . 

n-«o  n-wo  m*l  ' 

a 

Now  we  have  assumed  that  {XQ}  is  <f-irreducible,  so  from  Proposition  2  (ii), 
(A(x,  •)}  is  irreducible,  and  moreover  has  by  assumption  a  stationary 
measure  v.  Hence  (see  Proposition  4.2  of  [14]),  for  II-almost  all  x, 
and  all  B  e  & 

(3.14)  i  I  Am(x,B)  V(B)  . 

m-1 

Since  v  »  ip,  we  have  from  (3.14),  (3.13)  and  (3.9)  that  for  4-almost  all 
(i,x) 

lim  inf  <J>n(i,x)  £  v  (dw)  h(d,w)  <  0  . 

rr~  i 

a 

However,  we  can  emulate  the  proof  of  (9.11)  of  [14]  with  g(j,y)  *  j  to  show 
that  for  all  (i,x) , 

lim  inf  (|>n(i,x)  £  0  ; 
ir+«n 

this  shows  that  our  assumption  that  {X^}  does  not  have  a  stationary  probability 
measure  (and  specifically,  that  (3.4)  holds  for  individual  j)  cannot  be  true, 
and  so  II  does  exist  as  required.  □ 

The  proof  above  shows  that  Neuts'  condition  (3.1)  is  effectively  an 
average  "mean-drift"  condition,  where  the  averaging  is  done  over  the  distri¬ 
bution  v.  Theorem  9.2  of  [14]  shows  that  such  an  average  mean  drift 
condition  may  also  be  necessary,  under  some  extra  conditions,  for  positive 
recurrence. 

We  now  turn  to  the  necessity  of  condition  (3.1)  for  existence  of  a 
stationary  measure  for  the  chain  {Xn). 


i 
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In  order  Co  do  this,  we  first  define  a  further  chain  {X*}  with 
transition  probabilities  on  the  state  space  {...,-1, 0,1,2,...}  x  E  given 
by,  for  x  e  E,  B  e  &, 

P*(A,x;  j.B)  -  Aj  i+1(x,B)  ,  i  >  0,  j  -  ...,-1,0,1,... 

P*(i,x;  0,B)  -  6 (x,B)  ,  i  <  0  . 


The  chain  {X*}  has  the  same  motion  as  {X  }  except  that,  when  X  reaches  the 
n  n  n 

level  0,  X*  actually  takes  on  the  negative  level  that  X^  is  "trying"  to 

reach;  and  X*  is  then  replaced  at  level  0  at  the  next  step,  but  in  the 

same  "E-position".  Trivially,  if  {Xq}  is  ^irreducible  with  4>(0*E)  >  0 

then  so  is  {X*}.  We  also  have 
n 

Proposition  3  :  If  X  has  a  stationary  probability  measure  It,  then  {X* }  has 
a  stationary  probability  measure  II*,  and  for  some  constant  b,  we  have 
n*(i,A)  -  b  S(i,A) ,  i  >  0. 


Proof  :  The  stationary  equations  for  {X*}  are: 


(3.15) 


It* (J.B)  -  l  f  Jt*(k,dy)  P*(k,y;  j.B) 

k*°  J 

*  l  |  n*0My)  Ak_j+1(y»B>  *  i  >  0  j 


(3.16) 
and,  for  j 


-  0  , 


(3.17) 


n*(j,B)  -  ^II*(kfdy)  Ak_j+l(y’B)  *  J  <  0  » 

» 

oo  , 

H*(0,B)  -  l  II*(k,dy)  P*(k,y;  0,B) 
kMO  E 

OO  /  “1 

-  I  H*(k,dy)  A  (y,B)  +  [  n*(k,B) 

k-0  l  3  k— “ 

£ 

ao  r  oo 

-  I  JI*(k,dy)  l  A.  (y,B)  , 

k-0  i  l-k+1  1 
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using  (3.16). 

Now  (3.15)  and  (3.17)  are  the  stationary  equations  for  (X  };  these 

n 

have  the  solution  II(k,A),  and  this  is  unique  up  to  multiplication  by  a 
constant  b  ([.14],  Proposition  4.3).  From  (3.16)  we  can  construct  Il*(j,B) 
for  J  <  0,  once  we  have  II*(j,B)  for  j  >_  0,  and  the  proof  is  completed  by 
noting  that,  from  (3.16), 


Theorem  6  :  Suppose  {Xn>  has  an  invariant  measure  II.  Then  {A(x,*)}  has  an 
invariant  measure  v,  and  v  satisfies  (3.1)  provided 


(3.18) 


l  jH(j  ,E)  <  ® 
j-0 


Proof  :  From  Proposition  2  (iii),  v  exists  and  satisfies  v(B)  »  £  jl(k,B); 

k»0 

OB 

so  from  Proposition  3,  v(B)  -  b  £  II*(k,B).  Hence  (3.1)  will  hold  provided 

0 

W  f  00 

(3.19)  l  n*(k,dw)  (  l  P*(k,w;  j,E)  3  -  k]  <  0  . 

k«*0  4  j 

£ 

Now  let  h(3,x)  ■  j»  j  _>  0,  and  0  otherwise;  and  let 


U.  (k,w)  -  l  P*(k,w;  j ,dx)  h(3,x)  -  h(k,w) 
3—“ 

Clearly  (3.19)  will  hold  provided  we  can  prove 


(3.20) 


l  |  H*(k,dw)  U^(k,w)  <  0  . 


We  can  now  emulate  the  necessity  half  of  Theorem  9.2  of  [14)  to  deduce  that 
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(3.20) ,  as  in  (9.13)  of  [14],  always  holds  when  {X*}  has  a  stationary 

measure,  and  the  i  esult  will  be  proved.  The  only  condition  that  needs  to 
be  checked  is  that  the  interchange  of  integration  in  (9.14)  of  [14]  is 
valid,  and  it.  is  for  this  that  we  need  (3.18).  Q 

Remarks  :  (i)  The  need  for  a  bound  on  /  II(dy)  h(y)  in  (9.14)  of  [14]  was 
overlooked  in  the  conditions  given  there;  it  is  claimed  that  (9.6)  suffices 
but  this  not  in  fact  enough. 

(ii)  It  is  of  some  historical  interest  that  the  condition  (3.1) 
of  a  long-term  average  negative  mean  drift,  shown  above  to  be  essentially 
equivalent  to  positive  recurrence  of  {X^},  was  also  first  investigated  by 
Neuts  [5]  in  a  situation  where  only  two  "drift  values"  were  possible. 

It  is  possible  to  say  a  little  more  about  the  finiteness  of  the 
mean  (3.18)  of  the  measure  II.  We  have 

Proposition  4  :  The  stationary  measure  it  has  finite  mean  if  either 

(i)  the  measure  v  has  a  bounded  density  with  respect  to  the  measure  pit;  or 
(ii)  whenXQ  has  measure  git,  then  the  variance  of  the  return  time  to  level 
0  is  finite;  i.e. 

(3.21)  j  gii(dw)  t^Tg)  <  »  . 

E 

Proof  :  Using  the  operator-geometric  form  of  IT  given  by  Theorem  2,  we  have 
that  (3.18)  holds  provided 

•  >  l  3  C  J  0ii(dw)  SJ(W,E) 

1  E 

,  00  00 

-  c  Ji(dw)  l  l  S^(y,E) 

k-1  j-k 

oo  oo 

il(dw)  l  Sk(w,dy)  l  Sj(y,E) 


(3.22) 


Using  the  identification  of  v  in  Proposition  2  (iii)  shows  that  this  is 


(3.23)  "  v(dy)  l  Sj(y,E)  -  J  jl(j,E)  . 

E  3=0  j 

If  v  has  a  bounded  density  with  respect  to  qIT,  then  (3.23)  is  finite  since 

Y  CO 

Qn(dy)  l  Sj(y,E)  -  C  £  n(J,E)  <  c  ; 

l  i’O  i 

this  proves  (i) .  To  prove  (ii)  we  use  the  fact  that 

ao 

l  S^y.E)  -  1  +  1E(T  |  Xn  =>  (0,y>) 
j-0 

so  that  (3.22)  Is  also 

(3.24)  |  l  il(k,dy)  [1  +  E(Tq  j  XQ  -  0,y)] 

E  k”1 

<  1  +  [  l  il(k,dy)  E(tq  |  X0  -  k,y)  ; 

k*0  U 

E 

from  Cogburn  [2]  Corollary  3.1,  we  have  that  (3.24)  is  finite  provided  (3.21) 
holds .  □ 

In  this  section  we  have  proved  that  Neuts'  mean  drift  condition  is 
essentially  related  probabilistically  to  the  positive  recurrence  of  the 
chain  {XQ}.  It  Is  of  some  interest  to  know  what  the  positive  recurrence 
of  (x^  implies  for  an  arbitrary  chain  -f Xn J  with  structure  (1.1).  This 
seems  difficult  to  say.  From  (1.2)  and  (2.17),  L(x,E)  -  L(x,E),  and  this 
positive  recurrence  of  (Xn>  implies  at  least  that  {QXn}  is  well-defined  in 
the  sense  that  L(x,E)  “  1  for  Qll-almost  all  x.  Moreover  {y :  L(y,E)  ■  1} 
is  stochastically  closed,  so  provided  (Xn)  is  ^-irreducible  and 
Bk(y,*)  «  B^(y,»)  for  all  k,y,  we  can  deduce  that  L(x,E)  ■  1  for  4>-almost 
all  x. 

However,  this  does  not  even  guarantee  the  recurrence  of  the  process 
{qXq}.  The  following  example  illustrates  this. 


Example  :  Let  E  ■  {0,1,...},  and  let  the  transition  law  take  the  form  (1.1) 
with  only  Aq  and  A2  being  non-zero,  and  given  by 

Aq(x,x-1)  -  pq  ,  x  1  1  > 


Aq(x,x+1) 

■  (l-p)q  ,  x  >_  1 

and 

A0(0,0)  - 

pq,  Aq(0,1)  -  (l-p)q  5 

A2(x,x-1) 

-  p(l-q)  ,  x  >_  1 

A2(x,xfl) 

■  (1-p)  (1-q)  ,  x  _>  1 

and 

a2(o,o)  - 

p(l-q)  ,  A2(0,1)  -  (1-] 

with 

0  < 

:  p  <  1  ,  0  <  q  <  1  . 

Then  the  marginal  chain  with  transition  matrix  (A(x,*)}  Is  a  random 
walk  on  a  half-line,  with  a  stationary  probability  measure  v  if  and  only  if 
p  >  3S.  In  the  other  dimension  the  mean  change  in  level  is,  for  levels 
greater  than  zero,  always  of  the  same  sign;  and  this  is  negative,  i.e. 

B(x)  >  1,  if  and  only  if  q  <  Hence  the  necessary  and  sufficient 

condition  for  {Xq}  to  have  a  stationary  measure  is  q  <  <  p,  from  Theorems 

5  and  6. 

Let  us  alter  the  transition  law  to  define  (Xn>  by  setting  for  x  ^  1 

Bq(x,x-1)  -  r(l-q) 

Bq(x,x+1)  -  (l-r)(l-q)  , 

but  leave  B^(x,x-1)  ■  p(l-q),  B^(x,x+1)  ■  (1-p) (1— q)  as  for  {Xq}.  For  the 
embedded  chain  {qXr}  we  have,  by  considering  whether  the  chain  {Xn}  leaves 
level  zero  or  not,  for  x  _>  1, 

(3.25)  E(0Xq  |  QXn  -  x)  -  x  >  (l-2r)  (1-q)  +  (l-2p)  •  .  q  . 

the  second  term  comes  because,  once  the  chain  {Xn>  leaves  level  0  (with 
probability  q)  the  expected  number  of  steps  to  return  is  given  by  (l-q)/(l-2q) 
and  on  each  such  step  the  expected  drift  to  the  left  is,  independent  of  level 
change,  l-2p  (unless  the  chain  hits  (U*0),  which  gives  the  inequality  in 
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(3.25)). 

From  (3.25)  we  see  Chat,  provided  q  and  r  are  small  enough,  and  p  is 
not  too  much  greater  than  *5,  we  can  ensure  that  for  all  x  1 

®<0X„  I  0Xn  '  X)  -  *  ±  5  ’ 

This  is  enough  to  ensure  that  {qX^  is  transient,  as  can  be  seen  by 

comparing  {rtX  }  with  a  random  walk  with  positive  drift.  Hence  {_X  }  does 
u  n  u  n 

not  have  a  stationary  measure,  and  thus  neither  does  {x^} .  □ 

In  order  to  deduce  positive  recurrence  of  {X  }  we  need  to  know, 

n  t 

essentially,  that  qII  exists  and  has  a  bounded  density  with  respect  to  qII. 

If  this  happens  then  we  have,  for  some  <, 

|  Qn(dy)  E(tq  |  Xq  *  (0,y))  <  <  j  n(dy)  tE(TQ  |  XQ  *  (O.y)) 

-  K  |  Qn(dy)  E(T0  l  XQ  *  (O.y))  <  •  , 

and  so  {Xr}  is  positive  recurrent  also.  The  case  when  E  is  finite  is  the 
only  situation  where  this  is  automatically  true,  and  we  can  at  least  prove 

Proposition  5  :  If  E  is  finite  and  (A(x,*)}  is  an  irreducible  stochastic 
matrix,  then  (3.1)  is  necessary  and  sufficient  for  {X^}  to  have  a  stationary 
distribution.  *"  0 

Proof  :  From  the  argument  above  and  Proposition  4  (i),  our  result  will 
follow  from  Theorems  5  and  6  provided  only  that  qII(x)  >  0  for  each  x  e  E. 

It  is  easy  to  show  that  irreducibility  of  (A(x, •)}  Implies  irreducibility 
of  (L(x, •)},  which  gives  the  required  positivity.  □ 
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§4  Some  queueing  models 
(i)  The  GI/PH/1  queue 

This  model  is  described  in  detail  in  Neuts  [8],  and  we  do  not  repeat 

details  here.  The  bivariate  chain  {X  }  consists  of  (N  ,P  ),  where  N  is 

n  n  n  n 

the  number  of  customers  Immediately  before  the  ntk  arrival  and  P^  is  the 
phase  of  service  immediately  after  the  nc^  arrival. 

The  main  result  that  is  new  in  this  situation  comes  from  Proposition 
1.  Recall  that  the  chain  {qXq}  gives  the  phase  on  successive  returns  to 
level  (0,E);  this  is  precisely  the  phase  of  service  of  the  arriving 
customer  who  finds  the  queue  empty,  and  so  {gXn)  consists  of  a  sequence  of 
independent  and  identically  distributed  random  variables;  from  [ 8  ]  we  have 
that  the  distribution  of  these  phases  is  given  by  a  vector  a.  The  chain 
{qX^},  and  hence  {Xfl},  then  trivially  satisfies  the  irreducibility  Condition 
I,  with  <t(j)  «  a j .  Neuts  ([S'],  Lemma  7)  shows  that  (3.1)  is  equivalent  to 

the  usual  stability  condition  for  the  GI/PH/1  queue. 

From  the  discussions  of  §3  and  §2  it  follows  that,  since  E  is  finite, 
under  these  stability  conditions  the  invariant  measure  for  {X^}  has  the  form 

n(k,j)  -  c  l  am  sk(m,j)  ; 
m 

clearly  Qll(j)  ■  since  {qXq}  a  sequence  of  i.i.d.  variables.  Since  E 
is  finite,  S  is  R-recurrent  when  it  is  irreducible,  and  so  from  Theorem  4 
we  have  that  S  is  the  unique  solution  of  S  ■  A[S]  with  convergence  norm 
R  >  1  in  this  case.  This  shows  that  Theorem  4  of  [ 8  ]  follows  from  our 
results  without  extra  conditions  being  necessary. 

The  identification  of  Xq  -  ca  is  deduced  in  [ 8  )  algebraically,  but 
the  fact  that,  from  Proposition  1,  B[S]  *  a  also,  is  not  noticed  there. 


There  are  various  embedded  Markov  chains  on  which  the  results  above 


might  be  demonstrated.  In  order  to  compare  results  with  the  GI/PH/I  queue 
above,  we  will  take  {Xn)  “  {(Nn,Sn)}  as  the  chain  with  Nn  as  the  number  of 
customers  immediately  before  the  n^  interarrival  time  and  Sn  as  the  residual 
service  time  immediately  after  the  nC**  inter arrival  time. 

We  let  G  denote  the  distribution  function  of  service  times,  and 


CD 

-I 


write  y  *  t  dG(t)  for  the  mean  service  time;  we  let  F  denote  the  distri¬ 


bution  function  of  interarrival  times,  and  write  X  ** 


t  dF(t)  for  its  mean. 


We  assume  both  X  and  y  are  finite. 

Let  a^,a2,a3,...  denote  a  renewal  process  with  -  anl  having  the 
service  distribution  function  G;  and  let  denote  the  residual  life-time 
at  time  t  in  this  process,  i.e.  *  t  -  where  N(t)  is  the  number  of 

renewals  in  [0,t].  If  RQ  ■  x  then  *  x.  Now  write 


(4.D  Pn(x,y)  “  *(on  1  t  <  an+l»  Rt  1  y  I  R0  “  x) 

for  the  probability  that  n  renewals  occur  in  [0,x]  and  that  the  residual 

lifetime  at  t  is  in  [0,y]  given  Rq  *  x.  It  Is  easy  to  verify  that  the 

chain  {X  }  has  the  form  (1.1)  with 
n 

00 

(4.2)  An(x,[0,y])  -  |  P*(x,y)  dF(t) 

0 

and 

OD 

B  (x,lO,y])  -  I  l  A.(x, [0,»))]  G(y)  . 
n+1  J 


Hence  {qXq)  again  consists  of  i.i.d.  variables  with  distribution 
function  G,  and  so  QH[0,y]  •  G(y)  provided  it  exists ;  for  this  we  need  to 
ensure  that  {qXq)  does  not  terminate.  To  check  that  (qX^  is  a  proper 
chain  we  can  use  (3.1),  since  (qX^  terminates  if  and  only  if  {^X  }  does. 
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Our  main  result  is 


Theorem  7  :  (i)  A  necessary  and  sufficient  condition  for  {A(x, •)}  to  have 
a  stationary  probability  measure  v  satisfying  (3.1)  is  that  X  >  y. 

(ii)  When  X  >  y,  {X^}  has  an  invariant  probability  measure 
H(j.*)  given  by 

00 

(A. 3)  n(k,0  *  c  |  dG(x)  Sk(x,  •) 

0 

where  S(x, •)  is  the  minimal  solution  of 


!(x,[0,y])-I  jV(x,dw)  Aj  (w,  [0,y])  ,  ye  [0,<»)  ; 


and  the  constant  c  Is  given  by 
(A. 4)  c 


r  00  OO 

1  +  {  [I  Fn*(x) ]  dG(x) }  {exp  l  (1-a  )  /  n} 
l  n-0  n-1 


where 
(A.  5) 


a 

aQ  -  |  [1  -  l**(x>]  dG*n(x)  . 


Proof  :  (i)  From  (A.l)  and  (A. 2), 


(A. 6)  A(x,[0,y])  -  |  F(dt)  &  (x,y) 

where  Pt(x,y)  ■  TP(Rt  <_  y  |  Rg  ■  x) .  The  kernel  A  is  thus  the  transition 
probability  kernel  corresponding  to  the  residual  life-time  process  sampled 
at  points  of  an  (independent)  renewal  process  generated  by  F.  Provided 

y  <  •,  the  residual  life-time  process  has  invariant  measure  v  with 

x 

v[0,x]  -  i  |  [1  -  G(x)]dx  , 

0 

and  this  transfers  immediately  to  the  chain  with  kernel  (A. 6)  (see  [13]  for 
other  results  concerning  the  relationship  of  the  process  {Rc>  to  its 
"sampled”  version.) 
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On  Che  other  hand,  from  (4.1) 

00 

l  n  P*(x,«)  -  E(N(t)  |  Rfl  *  x)  . 
n“0 

The  stationarlty  of  V  then  gives,  by  Fubini's  Theorem 

CO  00 

J  V(dx)  S(x)  =  j  (  |  V(dx)  TE(N(t)  J  RQ  -  x) ]  dF(t) 

0  0 
00 

-  |  [t/yj  dF(t)  -  X/y  , 

0 

which  gives  our  first  result. 

(ii)  From  Theorem  5,  (Xn)  will  have  a  stationary  probability  measure 
when  X  >  y,  and  hence  {qXq}  will  at  least  return  to  £  with  probability  one. 
Hence  qII  exists  and  has  distribution  function  G(x) ,  and  II  given  by  (4.3)  is 
at  least  a  cr-finite  stationary  measure  for  {X^}  from  Theorem  1  (ii) . 

We  have  to  prove  that  H  is  a  probability  measure,  and  this  does  not 
follow  from  the  general  theory  used  so  far:  see  the  end  of  §3  for  a 
discussion  of  the  difficulties.  We  need  to  prove 

(4.7)  |  Qn(dy)  *(tq  |  XQ  -(0,y))  <  »  , 

and  to  do  this  we  need  to  evaluate  E(Tq  |  Xq  -  (0,y))  for  this  particular 
chain.  Suppose  Xq  ■  (0,y)  and  that  during  the  service  time  y  of  this 
first  arriving  customer,  n^  further  customers  arrive.  By  rearranging  the 
order  of  service  in  the  usual  way,  it  is  clear  that  in  this  situation  the 
expected  number  of  customers  served  before  £  is  again  reached  is  nyY,  where 
Y  is  the  expected  number  of  customers  in  a  busy  period  in  the  GI/G/1  queue; 
from  [4]  it  is  known  that,  when  X  >  y, 

00 

(4.8)  Y  *  exp{  l  (l-an)/n)  <  » 

n«l 


where  aQ  is  given  by  (4.5). 
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Thus 

TE(tq  |  XQ  -  (0,y) )  -  1  +TE(nyy  I  XQ  -  (0,y)) 

(4.9)  -l+[  l  Fn*(y)]y  , 

n=0 

and  standard  renewal  theory  [3]  shows  that  for  any  £  and  y  large, 

(4.10)  0  <  [I  Fn*(y)  -  y/X]  <  ye 

n=0 

Since  gll[0,y]  ■  G(y)  and  G  has  a  finite  mean,  (4.7)  follows  from  (4.9)  and 

(4.10) .  The  (finite)  value  (4.4)  of  (4.7),  which  is  the  constant  c  in 

(4.3),  follows  from  (4.8)  and  (4.9).  □ 

As  a  corollary  to  (4.10)  and  (4.11)  we  see  that,  if  {X^}  is  any  chain 

with  structure.' (1.1)  where  A  is  given  by  (4.2)  and  B  has  the  "replacement" 

n  n 

value 

00 

B’ (x, [0,yl)  -  [  l  A, (x, (0,”)) ]  H(y) 
n+1  J 

for  some  distribution  function  H,  then  (X'}  will  have  a  a-finite  invariant 

n 

measure 

00 

11’  (It,  (0,y])  -  j  dH(x)  sk(x,[0,y))  ; 

0 

and  II *  will  be  a  probability  measure  if  and  only  if  H  has  a  finite  mean. 

Ue  can  also  obtain,  for  the  queueing  chain  {Xq}, 

Proposition  6  :  The  stationary  distribution  II  has  finite  mean 

y*  ■  £  kll(k, [0,®))  if  and  only  if  F  has  finite  variance.  In  this  case 
k 

00 

(4.U)  U*  -  f  IX  -  0(x)H  I  »■*(*>]■»  4* 

“i  n-0 

where  y  is  given  by  (4.8). 
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Proof  :  From  (3.23)  we  have  that  y*  is  given  by  (4.11),  and  from  (4.10)  y* 
will  be  finite  if  and  only  if  V  has  a  finite  mean.  It  is  well-known  that 
this  holds  if  and  only  if  F  has  a  finite  variance.  □ 

(iii)  The  M/G/l  queue 

The  expression  [4.8]  for  y  simplifies  when  the  inter-arrival  time  is 
exponential:  from  Neuts  [7],  we  have 

y  -  A/ (A  -  y)  . 

In  this  case  also  we  have  the  expected  number  of  arrivals  in  [0,y]  is  just 
y/A,  and  so  from  (4.4),  c  =  A/(A  -  y)  also. 

This  is  a  computationally  useful  fact.  The  standard  method  of 
deriving  {II(k,A)}  from  the  theory  above  is  to  solve  the  non-linear  operator 
equation  S  -  A[S]  to  the  desired  accuracy  then  substitute  in  (4.3).  In 
general  c  can  only  be  estimated  as  the  normalising  constant 

c  -  [f  dG(x)  l  Sk(x,[0,co))]-1  f 
0  0 

where  N  is  the  number  of  terms  for  which  the  iterates  are  calculated. 

The  difference  [1  -  c/cjj]  represents  the  total  probability  mass  in  the 
levels  above  N,  i.e. 

00 

[i  -  c/cw]  -  l  n(j .to,-))  , 

N  N+l 

which  is  the  intrinsic  error  in  computation  of  II  when  only  a  finite  number 
of  the  iterates  S  are  used. 

Finally,  we  note  that  in  this  case,  from  (4.11)  we  can  recover  the 
fact  that  the  mean  queue  length  at  inter-arrival  times  in  equilibrium  is 

OO 

y*  -  J*[j  y2  dG(y)  ]  /  y(A-y)  . 

0 
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